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Abstract The current evaluation and comparison system 
of scientific production by impact factor has been criticised 
from different perspectives in recent years, and has ensured 
that publishing in high-impact journals does not necessarily 
imply that works are quality works. Many of these jobs 
are mostly not cited or, in the best of cases, only a very 
small number are, especially during the first years since their 
publication [4, 21]. In this context, the h-Index appeared 
in 2005 [19], and is currently one of the most bibliometric 
indicators used to estimate the success of research work 
and to predict the impact of future production. Following 
that proposed in [11], we used the h-index as a measure of 
scientific quality and productivity of mathematics teachers 
in Public Universities of the Valencian Community on the 
Web of Science. The results showed that the h-Index widely 
correlates citations and number of items. 
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1 Introduction 

Scientific research has greatly influenced the economic 
and social development of countries. In recent years, 
resources have declined with the global economic crisis, 
and the need to evaluate scientific activity and productivity 
has emerged, which is one of the most direct criteria in the 
scientific literature [9, 21, 7, 8]. We believe that scientific 
productivity evaluations are a necessary exercise because, 
in this way, researchers and universities can receive certain 
benefits through assessments made by institutional bodies 
(in Spain, Agencia Nacional de Evaluacin y Prospectiva 
ANEP, Comisin Nacional Evaluadora de la Actividad 
InvestigadoraCNEAI o Agencia Nacional de Evaluacin de la 
Calidad y Acreditacin, ANECA). The two main criteria used 
in Spain are, apart from the number of articles published 
(standard production) in the journals indexed on the (ISI) 
Web of Knowledge, the number of citations that the research 


has received and the impact factor of the journal that has pub¬ 
lished the research work (quality criterion) [21]. This evalua¬ 
tion and comparison system of scientific production by im¬ 
pact factor has been insistently criticised as publishing in 
high-impact journals does not necessarily imply that such 
works are quality works, and many of those jobs are mostly 
not cited or, in the best of cases, only a very small number 
are, especially during the first years since their publication 
[4, 6, 13, 21, 17]. A quite different work appeared half way 
through 2005, and was baptised by a physics teacher at the 
University of California, San Diego, as the h-index [11]. The 
idea is extremely simple. Each work by a given author is 
collected and sorted into descending order based on the ci¬ 
tations received. Therefore, each work occupies a position 
in a ranking according to the number of citations that it re¬ 
ceives, called a range, along with citations. Thus two lists 
of numbers are created, an upward one (ranges) and a down¬ 
ward (citations) one. Where both values cross, we obtain the 
h-index, a measure of position, namely one in which the vol¬ 
ume of citations is less than or equals the number of ordered 
items that occupies the downward distribution of citations 
[10]. Notwithstanding, it is true that a scientists h-index can 
never diminish over time [10]. According to Braun, Glnzel 
and Schubert [3], among the advantages that the h-index of¬ 
fers we find: (a) it is robust; that is, it is insensitive to any 
accidental excess from unmentioned works, and also to one 
or more extraordinarily cited works; (b) it combines the effect 
of’’quantity” (number of publications) and ’’quality” (citation 
rate) in a balanced and targeted manner; and (c) it can be cal¬ 
culated for specific periods, and not just for a lifetime. As far 
as we know, the h-index in the Mathematical Education area 
has never been studied in Spain. Therefore, the objectives of 
this study were to: present the h-index of the full-time teach¬ 
ers in different public universities in the Valencian Commu¬ 
nity (east Spain) in this knowledge area; examine the statisti¬ 
cal distribution of this index in this population, and see where 
the proportionality constant is located; consider the relation¬ 
ship between the h-index and other popular measures used in 
Spain to evaluate research productivity; e.g. number of arti¬ 
cles and citation index; compare the two areas of expertise to 
the journals that publish, the mean of authors per article and 
evolution on the ISI Web of Knowledge. 
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2 Method 

2.1 Design 

This is a descriptive, comparative and correlational study 
of document analyses. It took into account the considerations 
proposed by Hirsch [11], the considerations made by Hernan¬ 
dez et al. [19], Ramos-Alvarez et al. (2008), the recommen¬ 
dations in the Principles of Berlin for Rankings of Institutions 
of Higher Education [12] and the study of Olives-Avila and 
Musi-Lettuce [15]. 

2.2 Participants 

The sample consisted in 19 full-time university teachers 
(assistant doctors, university holders, contracted doctors, col¬ 
lege and university professorrs) who belong to the mathemat¬ 
ical education area in several universities in the Valencian 
Community (east Spain) 

2.3 Materials 

a) Websites of the departments in the Mathematical Educa¬ 
tion Area b) The (ISI) Web of Knowledge Database. This is 
available on the platform (ISI) Web of Knowledge and con¬ 
tains information about multidisciplinary research in journals 
for which their impact factor is calculated. 

2.4 Procedure 

Firstly, participants were selected from the information 
that appeared on the websites of the Mathematical Education 
Departments in Spain. Secondly, data were obtained accord¬ 
ing to the records collected on the (ISI) Web of Knowledge. 
We had to bear in mind that there were mistakes in many 
the names of authors (example: Author = (Gonzalez, Luis- 
Millan) AND Address = (Univ Valencia) / Author = (Gon¬ 
zalez, LM or Author = (Casajus, Jose A.) AND Author = 
(Casajus, JA)). It was found that the same author and the 
same cited work had been collected with different names. For 
a more detailed review of the process, it is worth referring to 
the study of Osca-Lluch, Civera and Pearanda [16], where 
possible variations are mentioned in the names of teachers 
affiliate institutions. Others tended to attach their names by a 
script, so this was also taken into account in the search. The 
data collection process was performed in the above database 
by entering the section ’’General Search”, where the fields 
’’Author” and ’’Address” were used, and the ’’Topic” field was 
even used in some cases. One surname, or both surnames as 
in Spain, were always inputted, and sometimes with a capital 
letter, other times in the lower case, and at times followed by 
a space and the initials of the authors name. Wildcards and 
Boolean operators were also used to ensure the delivery of all 
items by authors in order to consider all the possible combi¬ 
nations: the initial(s) of the first name(s) and the initial of the 
first surname, followed in all cases by the name supplied by 
the official university websites. 

2.5 Method to calculate the H-index 

The method to calculate the h-index that Hirsch suggested 
[11], which was described in more detail by Imperial and 
Rodriguez-Navarro [20], consists in the following steps: log¬ 
ging into the Web of Knowledge, choosing the ’’General 


Search” option, and keying in the authors name in the cor¬ 
responding window. Currently after having sought the au¬ 
thor, the (ISI) Web of Knowledge allows us to immediately 
calculate the h-index, create a report of citations, and the fol¬ 
lowing items are obtained almost instantaneously: total times 
cited, total times quoted without self-citations, articles cited 
in, total articles without self-citations, average citations per 
element and h-index. 

2.6 Total number of citations and maximum citation 

The total number of citations is the sum of all the citations 
of a researchers work in the aforementioned database. Max¬ 
imum citation is a work that has been cited the most among 
the works of a given author. 

Searches were made from October 2015 to November 
2015. Finally, we ought to consider that there could be some 
variations when viewing the results in this study. This is 
mainly because the database (ISI) Web of Knowledge does 
not report monthly reports with static results, but is instead 
characterised as being dynamic as information is constantly 
being added. Hence the number of records may vary from 
day to day. 

3 Results and data analysis 

We studied 19 teachers. We considered those who occu¬ 
pied full-time posts in any public Valencian university and in 
the Mathematical Education area. Of these, 17 had publica¬ 
tions on the Web of Knowledge (89.5%). 

3.1 h-index for each teacher 

Figure 1 shows a chart used to visualise the two different 
h-indices considered for all the 19 Mathematical Education 
teachers. We can see that almost all of them had a general h- 
index of their publications, which was higher than or equalled 
that which corresponded to Mathematical Education. This 
indicates that some started out in some mathematics branch 
other than Mathematical Education. The mean h-index in 
Mathematical Education in the Valencian Community was 
0 . 68 . 
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Figure 1 . Comparison between the general h-index and that corresponding 
to Mathematical Education of the studied teachers. 
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3.2 Articles and h-index for each teacher. 

Figure 2 shows the relation between the number of articles 
published and the h-index. We stress that only two teachers 
had an h-index over 3. The mean number of articles for each 
studied teacher was 6.4. However, it is necessary to point 
out that of the 114 articles found, 50 belonged to the two 
aforementioned teachers. 
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Figure 2. Comparison between the general h-index of the studied teachers 
and the number of published articles. 
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Figure 4. Comparison between the Mathematical Education h-index of the 
studied teachers and the number of citations (without self-citations) that their 
articles had received. 
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3.3 Citations and h-index for each teacher. 

When we considered only citations in Mathematical Ed¬ 
ucation, the rank of citations fell between 0 and 67 when 
considering the citations of any publication (Figure 3), and 
between 0 and 37 when not considering self-citations (Fig¬ 
ure 4). The largest number of citations corresponded to the 
highest h-index in Mathematical Education. 

80 
70 
60 

§ 50 

g 40 

5 30 
20 
10 
0 

0 1 2 3 4 5 6 


Universities: 62 UA, 67 UV, 71 UJI 
Figure 5. Comparison between h-indices and the considered universities. 


Education, the situation reversed (Figure 6): the UA had an 
index of 5, while the UV had one of 2. 

When we observed the articles that we found for each stud¬ 
ied teacher, the UA excelled with 36 and 14 publications for 
two of its teachers, whereas the UV had a maximum of 10 
publications for one of its teachers. In absolute values, we 
found 114 articles in the Valencian Community, 57 from the 
UV and 55 from the UA (Figure 7). 

Finally, the citations received (after deleting self-citations) 
varied between 0 and 121. Of the 336 citations we detected 
in the articles, 282 corresponded to the UV, whereas 53 were 
for the UA (Figure 8). 
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Figure 3. Comparison between the Mathematical Education h-index of the 
studied teachers and the number of citations that their articles had received. 
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Once again, 99% of the citations concentrated on four pro¬ 
fessors, of whom two were those previously mentioned. 

3.4 Analysis by Universities. 

The three studied universities with a Mathematical Educa¬ 
tion area present on their own website were the Universitat 
Jaume I of Castelln (UJI), the University of Valencia (UV) 
and the University of Alicante (UA). According to universi¬ 
ties, the UV had higher h-indices than the other two (7), fol¬ 
lowed by the UA with a maximum of 5 (Figure 5). When we 
observed only the h-index that corresponded to Mathematical 
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Figure 6. Comparison between the Mathematical Education h-index and the 
considered universities. 
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people, and are acceptably high. All this has to be taken into 
account for the qualifications of teachers in these knowledge 
areas because, as their research career has been neither well 
established nor lasted many years, starting ones productivity 
in Mathematical Education is not easy. Therefore, we believe 
that this study is useful in many ways, some of which are: 
for teachers because they have a standard of comparison; for 
the various committees and commissions that evaluate teach¬ 
ers in this knowledge area; finally, this type of study allows 
those who legislate to adapt distinct levels of accreditations 
to different knowledge more realistically. 


Figure 7. Comparison between the Mathematical Education h-index and the Acknowledgements 
considered universities. 
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We are very grateful to experts for their appropriate and 
constructive suggestions to improve this template. 
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4 Conclusions 
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